Markov Decision Models with Weighted Discounted Criteria

We consider a discrete time Markov Decision Process with innnite horizon. The criterion to be maximized is the sum of a number of standard discounted rewards, each with a diierent discount factor. Situations in which such criteria arise include modeling investments, production, modeling projects of diierent durations and systems with multiple criteria, and some axiomatic formulations of multi-attribute preference theory. We show that for this criterion for some positive there need not exist an-optimal (randomized) stationary strategy, even when the state and action sets are nite. However,-optimal Markov (non-randomized) strategies and optimal Markov strategies exist under weak conditions. We exhibit-optimal Markov strategies which are stationary from some time onward. When both state and action spaces are nite, there exists an optimal Markov strategy with this property. We provide an explicit algorithm for the computation of such strategies and give a description of the set of optimal strategies.